Tree-Based Methods for Fuzzy Rule Extraction

This paper is concerned with the application of a treebased regression model to extract fuzzy rules from highdimensional data. We introduce a locally weighted scheme to the identification of Takagi-Sugeno type rules. It is proposed to apply the sequential least-squares method to estimate the linear model. A hierarchical clustering takes place in the product space of systems inputs and outputs and each path from the root to a leaf corresponds to a fuzzy IF-THEN rule. Only a subset of the rules is considered based on the locality of the input query data. At each hierarchy, a discriminating subspace is derived from the high-dimensional input space for a good generalization capability. Both a synthetic data set as well as a real-world robot navigation problem are considered to illustrate the working and the applicability of the algorithm. Introduction The fuzzy rule-based model represents a simple and powerful tool to model system dynamics by means of IF-THEN rules. Human experts’ knowledge is usually used to design those rules. This acquisition process is cumbersome, and in some cases (e.g., unknown system), expert knowledge is not available. Neuro-fuzzy approaches (Lin & Lee 1996) are introduced with the purpose of extracting the fuzzy rules, and the membership-functions automatically from measured input-output pairs. However, in neural-fuzzy approaches, the dimension of the input vector is usually small in order to be manageable. In this paper, we propose a hierarchical fuzzy model, which extracts rules from high-dimensional data pairs. The major distinctions of the model include: First, we derive automatically discriminant feature subspaces in a coarse-tofine manner from the high-dimensional input space. The features are most discriminative in the sense that input variables irrelevant to the output are disregarded to achieve better generalization. Second, we organize the rule base in a hierarchical way. This tree architecture recursively excludes many far-away, unrelated rules from consideration; thus, the time for inference and learning is O(log(N)), where N is the size of the rule base. Copyright c © 2005, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. The remainder of this paper is organized as follows: Section presents the fuzzy model. We then extend it to the hierarchical form in Section . The results of the simulation and real robot experiments are reported in Section . Discussions and concluding remarks are given in Section . Fuzzy Modeling Fuzzy Rule-Based Systems A fuzzy rule is a syntactic structure of the form IF antecedent THEN consequent (1) where each antecedent and consequent are well-formed fuzzy predicates. Given a system with the input vector x ∈ X ⊂ R and the output vector y ∈ Y ⊂ R. In the Takagi-Sugeno (TS) model (Takagi & Sugeno 1985), the base of r fuzzy rules is represented by IF x is A1 THEN y1 = β 1 z + β0,1 .. IF x is AN THEN yr = β N z + β0,N (2) where A1,...,AN denote multivariate antecedents (fuzzy predicates) defined on the universe R. Here the consequents are linear function of the dependent vector z that is the projected vector on a discriminating subspace (see Section ). The membership function of the i-th antecedent Ai is defined as Ai(x) : R 7→ [0, 1] (3) For a query input x the output of the rule-base is calculated by aggregating the individual rules contributions y = ∑N i=1 Ai(x)yi ∑N i=1 Ai(x) (4) where yi and Ai(x) denote the output and the activation level of the i-th rule, respectively. Incremental Parameter Estimation Two phases are involved to estimate the parameters of the ith rule. First, in structure identification phase the fuzzy predicate Ai is determined, which will be discussed in Section . Second, In the parameter identification phase, we assume the antecedent predicate Ai is fixed and apply sequential leastsquare algorithm to estimate the parameters: βi and β0,i of the rule. For notation simplicity we neglect rule index i. Consider a collection of t input-output data pairs (xk, yk), k = 1, 2, ..., t where xk is the n dimensional input vector and yk denotes the scalar target output of the system. Let zk, k = 1, ..., t be extracted feature vector which we will discuss in Section . As in locally weighted learning (LWR) (Atkeson, Moore, & Schaal 1997), we compute the following diagonal activation matrix: W = diag [A(x1) A(x2) ... A(xt)] with each one corresponding to the data pair. Let Z = [ z1 ... z T t ]T . For computational and analytical simplicity, let Ze = [Z 1] and θ = [ β β0 ]T . We formulate the estimation as finding the parameter θ such that the rule output, y, is